High AI training costs have been a significant barrier to AI adoption, preventing many companies from implementing AI technology. According to a 2017 Forrester Consulting Report, 48% of companies highlighted high technology costs as one of the primary reasons for not implementing AI-driven solutions. However, recent developments have shown that AI training costs are rapidly declining, and this trend is expected to continue in the future. According to the ARK Invest Big Ideas 2023 report, training costs of a large language model similar to GPT-3 level performance have plummeted from $4.6 million in 2020 to $450,000 in 2022, a decline of 70% per year. Let's explore this trend of declining AI training costs further and discuss the factors contributing to this decline.

It's always good to have concrete examples of technological progress, especially since macroeconomic stats aren't great at capturing the phenomenon. Enter: The Stanford Institute for Human-Centered Artificial Intelligence. It recently released its 2022 AI Index Report, which surveys the current landscape of AI research and industrialization. Among the stats tracked in this data-driven report is the cost to train AI systems like ImageNet, an application for identifying and categorizing visual objects. The report finds that training an image-classification system cost only $4.60 in 2021, compared to over $1,000 for a similar system in 2017: "In four short years, image classification training costs have decreased by a factor of 223."

Google researchers recently published a paper describing a framework -- SEED RL -- that scales AI model training to thousands of machines. They say that it could facilitate training at millions of frames per second on a machine while reducing costs by up to 80%, potentially leveling the playing field for startups that couldn't previously compete with large AI labs. Training sophisticated machine learning models in the cloud remains prohibitively expensive. According to a recent Synced report, the University of Washington's Grover, which is tailored for both the generation and detection of fake news, cost $25,000 to train over the course of two weeks. OpenAI racked up $256 per hour to train its GPT-2 language model, and Google spent an estimated $6,912 training BERT, a bidirectional transformer model that redefined the state of the art for 11 natural language processing tasks.